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Abstract: In this paper we prove large and moderate deviations 
principles for the recursive kernel estimator of a proba- 
bility density function and its partial derivatives. Un- 
like the density estimator, the derivatives estimators 
exhibit a quadratic behaviour not only for the moder- 
ate deviations scale but also for the large deviations 
one. We provide results both for the pointwise and 
the uniform deviations. 
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1 Introduction 



Let Xi, . . . ,X n be a sequence of independent and identically distributed Revalued random 
vectors with bounded probability density /. Let (h n ) be a positive sequence such that 
lim^oo h n = and h% — oo; the recursive kernel estimator of / is defined as 



where the kernel K is a continuous function such that limi^n^+oo K{x) = and f Rd K(x)dx = 
1. The estimate (1) is a recursive version of the well-known Rosenblatt kernel estimate (see 
Rosenblatt (1956) and Parzen (1962)); it was first discussed by Wolverton and Wagner 
(1969), Yamato (1971), and Davies (1973). The estimator (1) is easily updated each time 
an additionnal observation becomes available without resorting to past data, through the 
recursive relationship 



The weak and strong consistency of the recursive estimator of the density was studied by 
many authors; let us cite, among many others, Devroye (1979), Menon, Prasad and Singh 
(1984), and Wertz (1985). The law of the iterated logarithm of the recursive density estima- 
tor was established by Wegman and Davies (1979) and Roussas (1992). For other works on 
recursive density estimation, the reader is referred to the papers of Wegman (1972), Ahmad 
and Lin (1976), and Carroll (1976). 

Recently, large and moderate deviations results have been proved for the Rosenblatt 
density estimator and its derivatives. The large deviations principle has been studied by 
Louani (1998) and Worms (2001). Gao (2003) and Mokkadem, Pelletier and Worms (2005) 
extend these results and provide moderate deviations principles. The large and moderate 
deviations of the derivatives of the Rosenblatt density estimator are given in Mokkadem, 
Pelletier and Worms (2005). The purpose of this paper is to establish large and moderate 
deviations principles for the recursive density estimator /„ and its derivatives. 

Let us recall that a R m -valued sequence (Z n ) n >i satisfies a large deviations principle 
(LDP) with speed (v n ) and good rate function / if: 

(a) (u n ) is a positive sequence such that (u n ) j oo; 

(b) I : R m — > [0, oo] has compact level sets; 

(c) for every borel set B C M. m , 




(1) 




inf I(x) < lim inf v~ x log P [Z n e B] 

° n^oo 
xeB 

< lim sup v~ x logP [Z n e B] < - mS_I{x), 
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where B and B denote the interior and the closure of B respectively. Moreover, let (v n ) be 
a nonrandom sequence that goes to infinity; if (v n Z n ) satisfies a LDP, then (Z n ) is said to 
satisfy a moderate deviations principle (MDP). 

For any d-uplet [a] = (a±, . . . , a^j G N d , set \a\ — a± + • • • + ad, let 

denote the [a]-th partial derivative of / (if \a\ = 0, then d^f = f) and, for any j e N, let 
D&f denote the j-th differential of /. The recursive kernel estimator of the [a]-th partial 
derivative of / is defined as 



1 n 1 / 



x - Xi 



hi 

where the kernel K is chosen such that d^K ^ and the bandwidth such that J2 n /in +2 ' Q ' 



oo. 



Our first aim is to establish pointwise LDP for the recursive kernel density estimator /„. 
It turns out that expliciting the rate function in this case is more complex than either for 
the Rosenblatt kernel estimator, or for the derivatives estimators. That is the reason why, in 
this particular framework, we only consider bandwidths defined as (h n ) = (cn~ a ) with c > 
and a e]0, l/d[. We then prove that the sequence (f n (x) — f( x )) satisfies a LDP with speed 
(Xir=i hf) an d ra ^ e function 

where I(t) is the Fenchel-Legendre transform of the function ip defined as follows: 

ip(u) = [ s- ad (e sad T^ K ^ - l) dsdz. 



Our second aim is to provide pointwise LDP for the derivative estimators d^f n (with 
|a| > 1). In this case, we consider more general bandwidths defined as h n = h(n) for all n, 
where h is a regularly varying function with exponent (—a), a e]0, 1/ (d + 2|a|) [. We prove 
that the sequence 

(cH/„(z) - dWf(x)) 

satisfies a LDP of speed (Yl™ =1 h d+2 ^ j and quadratic rate function J[ Q ] : R — > R defined by 

if f(x)^0, J [a] , x .t~ 2f{x)kd[d[a]K{z) f dz (2) 
if f(x) = 0, J[ a ],x(0) = and J [a ], x (t) = oo for t ^ 0. 
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Our third aim is to prove pointwise MDP for the density estimator and for its derivatives. 
For any rf-uplet [a] such that |a| > 0, any positive sequence (v n ) satisfying 



v 2 



lim v n = oo and lim " —. = 0, 

n^oo n->oo 1 h ^ 

and general bandwidths (h n ), we prove that the random sequence 

V n {d [a] f n (x)-d^f(x)) 

satisfies a LDP of speed (Y™ =1 h d+2 ^ /v£j and rate function J[ a ],x(-) defined by Equation (2). 

Finally, we give a uniform version of the previous results. More precisely, let U be a 
subset of R d ; we establish large and moderate deviations principles for the sequence 
(sup xeU \d^f n (x) — <9' a '/(x)|) in the case either U is bounded or all the moments of / are 
finite. 

2 Assumptions and Results 

2.1 Pointwise LDP for the density estimator 

The assumptions required on the kernel K and the bandwidth (h n ) are the following. 

(HI) K : M. d — > H. is a bounded and integrable function , J Rd K(z)dz = 1 and limpn^oo K(z) = 
0. 

(H2) h n = crT a with < a < 1/d and c> 0. 

Before stating our results, we need to introduce the rate function for the LDP of the density 
estimator. Let tp : M — > K and / : R — > R be the functions defined as: 



) = / s' ad ( e sad T^ K ^ - l) dsdz and /(*) = sup {ut - ^(u)} 



(where s e [0, 1], z e R d ) and set 

S+ = {xe R d ; K(x) > 0} and S_ = {iG M. d ; K(x) < 0} . 

The following proposition gives the properties of the functions ip and /; in particular, the 
behaviour of the rate function /, which differs depending on whether K is non-negative or 
not, is explicited. 

Proposition 1 Let A be the Lebesgue measure on R d and let Assumption (HI) hold. 

(i) ip is strictly convex, twice continuously differentiable on R 7 and I is a good rate function 
on R. 
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(ii) If X(S-) = 0, then I(t) = +00 when t < 0, 1(0) = \(S + )/(l — ad), I is strictly convex 
on R and continuous on ]0, +00 [, and for any t > 

i(t)=t(^r 1 (t)-^((^r 1 (t)). (3) 



(Hi) If X(S-) > 0, then I is finite and strictly convex on R and (3) holds for any t e R. 
(iv) In both cases, the strict minimum of I is achieved by 1(1/(1 — ad)) = 0. 

Remark The following relations are straightforward, and will be used in the sequel: 

T(f \ f sup M>0 {wt-^(w)} if t> 1/(1 -ad) ( ] 

K > ' \ sup u<0 {ut - ^j(u)} if t< 1/(1 -ad). y> 

We can now state the LDP for the density estimator. 
Theorem 1 (Pointwise LDP for the density estimator) 

Let Hypotheses (H1)-(H2) hold and assume that f is continuous at x. Then, the sequence 
(fn(x) — f(x)) satisfies a LDP with speed (J27=i hf) an( ^ ra ^ e function defined as follows: 

if f(x)^0, I x :t^f(x)(l-ad)l[^ + m ^) 
if f(x) = 0, I x (0) = and I x (t) = +00 for t ^ 0. 

2.2 Pointwise LDP for the derivatives estimators 

Let [a] be a d-uplet such that \a\ > I. To establish pointwise LDP for d^f n , we need the 
following assumptions. 

(H3) h n = h(n) where the function h is locally bounded and varies regularly with exponent 
(-a), 0<a< l/(d + 2|a|). 

(H4) i) K is I a I -times differentiate on R d and limi^n^oo \\D^K(x) \\ = for any j e 

{o,...,H-i}. 

ii) d^K : R d — > R is a bounded and integrable function and J Rd [d^K(x)] dx 7^ 0. 

(H5) / is I a I -times differentiable on R d and its j-th differentials D^f are bounded on R d 
for any j G {0, . . . , \a\ — 1}. 

Remark A positive (not necessarily monotone) function L defined on ]0, 00 [ is slowly 
varying if lim^^ L(tx)/L(t) — 1; a function G is said to vary regularly with exponent p, 
p E R, if and only if there exists a slowly varying function L such that G(x) = x p L(x) 
(see, for example, Feller (1970) page 275). Typical examples of regularly varying functions 
with exponent p are x p , x p \ogx, x p loglogx, x p log re/ log logx, and so on. An important 
consequence of (H3) which will be used in the sequel is: 

n 

if (3a < 1, then lim — - V /if = -. (5) 

n^oo nfrP l-a(3 
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Theorem 2 (Pointwise LDP for the derivatives estimators) 

Let \a\ > 1 and assume that (HI), (H3)-(H5) hold and that d^f is continuous at x. Then, 
the sequence (d^f n (x) — d^f(x)) satisfies a LDP with speed (X^=i ^ +2 '°') an d ra ^ e func- 
tion J[ a ],x defined by (2). 

2.3 Pointwise MDP for the density estimator and its derivatives 

Let (v n ) be a positive sequence; we assume that 

v 2 

(H6) hm^oo v n = oo and hm — — " = 0. 

(H7) i) There exists an integer q > 2 such that Vs G {l,...,g — l},Vj G 
y s j K(y)dy j = 0, and / < oo. 



ii) lim ^VM = 0. 

i=l 

iii) <9 [a1 / is g-times differentiable on M d and M g = sup xeM<l \\D q d [a] f(x)\\ < +oo. 



Remark When h n = 0(n~ a ), with < a < l/(d + 2|a|), (H6) and (H7)ii) hold for 

instance for (v n ) = (n b ) for any b G]0,min{ag; (1 — a(d + 2|a|)) /2}[. 

The following theorem gives the MDP for the density estimator and its derivatives. 

Theorem 3 (Pointwise MDP) 

When \a\ = 0, let Assumptions (HI), (H2), (H6) and (HI) hold; when \a\ > I, let (HI), 
(H3)-(H7) hold. Ifd^f is q-times differentiable atx, then the sequence (v n (d^f n (x) — d^f(x))) 

satisfies a LDP with speed (X^Li hf +2 ^ /v 2 ) and good rate function J[ a ], x defined in (2). 



2.4 Uniform LDP and MDP for the density estimator and its 
derivatives 

To establish uniform large deviations principles for the density estimator and its derivatives, 
we need the following additionnal assumptions: 

(H8) i) There exists £ > such that J Rd \\x\\^f(x)dx < oo. 
ii) / is uniformly continuous. 

(H9) i) d^K is Holder continuous. 

ii) There exists 7 > such that z 1— > \\z z) is a bounded function. 

(H10) lim "> g( ^"| = and lim % = 0. 

(Hll) i) There exists ( > such that f Rd \\z\\^ \K(z) \ dz < oo. 

ii) There exists rj > such that z i— > ^^^^/(z) is a bounded function. 
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Remark When h n = O (n' a ) with a e]0, + 2|a|)[, (H10) holds for instance with 
(v n ) = (n 6 ) for any b e]0, (1 - a(d + 2\a\)) /2[. 

Set U C M d ; in order to state in a compact form the uniform large and moderate de- 
viations principles for the density estimator and its derivatives on U, we consider the large 
deviations case as the special case (v n ) = 1 and we set: 



9u(6) 



U>00 (1 - ad)I + y^J^^ ) if H=0 and (v n ) = 1 

5 2 (l-a 2 (<i+2|a|) 2 ) . 

: ->r- — otherwise, 



<7t/(5) = min{^((5),^(-5)}, 
where ||/||c/,oo = su P;EgC/ |/(a;)|. 



Remark The functions gu{) and gu{ ) are non-negative, continuous, increasing on ]0, +oo[ 
and decreasing on ] — oo, 0[, with a unique global minimum in (<jfi/(0) = gu(0) = 0). They 
are thus good rate functions (and gu(-) is strictly convex). 

Theorem 4 below states uniform LDP and MDP for (d^f n — d^f) on U in the case U 
is bounded, and Theorem 5 in the case U is unbounded. 

Theorem 4 (Uniform deviations on a bounded set) 

In the case \a\ = 0, let (HI), (H2), (HI), (H9)i), and (H10) hold. In the case \a\ > 1, let 
(H3)-(H5), (HI), (H9)i) and (H10) hold. Moreover, assume either that (v n ) = 1 or that (v n ) 
satisfies (H6). Then for any bounded subset U ofM. d and for all 5 > 0, 



lim n . 

ra^oo h d+2 \ a \ 



En i a 
i=i K 



logP 



supv n \dWf n (x)-dWf(x)\>5 
xeu 



-9u{S). 



(6) 



Theorem 5 (Uniform deviations on an unbounded set) 

Let Assumptions (HI), (Hl)-(Hll) hold. Moreover, 

• in the case \a\ = and (v n ) = 1, let (H2) hold; 

• in the case \a\ > 1 and (v n ) = 1, or \a\ > and (y n ) satisfies (H6), let (H3)-(H5) 
hold. 



Then for any subset U of R and for all 5 > 0, 



—9u{$) < liminf 



logP 



Ere ,d+2\a\ 
V 2 



supv n \dWf n (x)-dWf(x)\ >S 
wj>v n \dWf n (x)-dMf(x)\ >5 



xeu 



< 



£ + d 



~9u(6). 



The following corollary is a straightforward consequence of Theorem 5. 
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Corollary 1 

Under the assumptions of Theorem 5, if f Rd \\x\\^f(x)dx < oo G 1, then for any subset U 
ofR d , 



lim ^^logP 

n ^oo s^n h d+2\a\ 



svLpv n \dWf n (x)-dWf(x)\ >5 



-9u{8). (7) 



Comment Theorem 4 and Corollary 1 are LDP for the sequence (sup xel / \f n (x) — f(x)\). 
As a matter of fact, since the sequence (sup^^ \f n (x) — f(x)\) is positive and since g v is 
continuous on [0, +oo[, increasing and goes to infinity as 5 — > oo, the application of Lemma 
5 in Worms (2001) allows to deduce from (6) or (7) that (sup^g^ \f n (x) — f( x )\) satisfies a 
LDP with speed (X^Li ^f) anc ^ g°°d ra te function gu on R+. 



3 Proofs 



Let (Wn ) and (Bn) be the sequences defined as 



-E 



d [a] K 



x - Xj 
hi 



BW(x) = E[8Wf n (x)]-dWf(x). 

We have: 

d [a] f n (x) - dWf(x) = tfM(x) + B^\x). 
Theorems 1, 2 and 3 are consequences of (8) and the following propositions. 
Proposition 2 (Pointwise LDP and MDP for ) 



(8) 



1. Under the assumptions (HI) and (H2), the sequence (f n (x)—'E(f n (x))) satisfies a 
LDP with speed (X^=i ^f) an d rate function I x . 



2. Let \a\ > 1 and assume that (H3), (H4) hold, then the sequence \^$f l n '(x)J satisfies a 
LDP with speed (Y™ =1 h d+2 ^ j and rate function J[ a ], x - 

3. When \a\ = 0, let Assumptions (HI), (H2) and (H6) hold and when \a\ > 1, let 
(H3), (H4) and (H6) hold, then the sequence ^v n ^°\x)j satisfies a LDP with speed 

(j2i=i h i +2 ^/ v l) and rate function J [a ] )X . 

Proposition 3 (Uniform LDP and MDP for (*^* ] ) ) 

1. In the case \a\ = 0, let (HI), (H2), (H9)i) and (H10) hold. In the case \a\ > 1, let 
(H3)-(H5), (H9)i) and (H10) hold. Moreover, assume either that (v n ) = 1 or that (v n ) 
satisfies (H6); then for any bounded subset U ofM. d and for all 5 > 0, 



lim ^^logP 



Er=i K 



supv n \d [a] ^ n (x)\ > 5 

xeu 



= -~9u(S). 
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2. Let Assumptions (HI), (H8)-(H11) hold . Moreover, 

• in the case \a\ — and (v n ) = 1, let (H2) holds, 

• in the case either \a\ > 1 and (v n ) = 1, or \a\ > and (v n ) satisfies (H6), let 
(H3)-(H5) hold. 

Then for any subset U ofM d and for all 5 > 0, 



~9u($) < liminf 



vt 



n id+2\a\ 



< lim sup 



n ,d+2\a\ 



logP 



logP 



supv n \d^ n (x)\ > 5 



xeu 



supv n \d^ n (x)\ >5 



xeu 



< 



Proposition 4 (Pointwise and uniform convergence rate of B^ ) 

Let Assumptions (HI), (H3)-(H5) and (H7)i) hold. 

1) Ifd^f is q-times differentiable atx, then 

E (d [a] f n (x)) - d [a] f(x) = O (^f^- 

2) If (Hl)m) holds, then: 

n M f 

lim sup |E {d^Ux)) - dWf(x)\ < -1 / 



\z\\ q \K(z)\dz. 



Set x G M. d ; since the assumptions of Theorems 1 and 2 guarantee that lim^oo B^\x) = 0, 
Theorem 1 (respectively Theorem 2) is a straightforward consequence of the application of 
Part 1 (respectively of Part 2) of Proposition 2. Moreover, under the assumptions of Theo- 
rem 3, we have, by application of Part 1 of Proposition 4, lim^oo v n B^ (x) = 0; Theorem 3 
thus straightfully follows from the application of Part 3 of Proposition 2. Finally, Theorems 4 
and 5 are obtained by applying Proposition 3 together with the second part of Proposition 4. 

We now state a preliminary lemma, which will be used in the proof of Propositions 2 and 
3. For any hGR, set 



K,x( u ) 



mi id+2\a\ 



logE 



Er=i k 

f(x)(l-ad)[ij>{u)- 



sr^n ,d+2\a\ 

exp | u * 



u 



u 



1 — ad / 
f{x) [ [d [a] K(z)Ydz. 



2(1 -a 2 (d + 2\a\) 2 ) 
Lemma 1 ( Convergence of \ n , x ) 

• In the case \a\ = and (v n ) = 1, let (HI) and (H2) hold; 
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• In the case either \a\ > 1 and (v n ) = 1, or \a\ > and (v n ) satisfies (H6), let (HI), 
(H3) and (H4) hold. 

1. (Pointwise convergence) 

If f is continuous at x, then for all net, 

lim A n>x (u) = A x (u) (9) 



where 
A x (u) 



A^(-u) when v n = 1 and \a\ = 

A^(-u) when either v n — > oo and \a\ > or v n = 1 and |a| > 1. 



2. (Uniform convergence) 

If f is uniformly continuous, then the convergence (9) holds uniformly in x 6 [/ . 

Our proofs are now organized as follows: Lemma 1 is proved in Section 3.1, Proposition 2 
in Section 3.2, Proposition 3 in Section 3.3 and Proposition 4 in Section 3.4. Section 3.5 is 
devoted to the proof of Proposition 1 on the rate function I. 



3.1 Proof of Lemma 1 

Set u e R, Yi = d^K (^f) and a n = J2ti lh +2H . We have: 



K,x{u) 



^logE 



= -^logE 

a n 



exp ( u—^\x) 



^logE 

By Taylor expansion, there exists c i>n between 1 and E 
log(E 



cxp 



ua r 



nv n H i 



d+\a\ 



Y; 



1 

1=1 n i 



such that 



ua r 



= E 



cxp 



exp 



d+\a\ 



Y 



nv n K % 

ucin y I — 1 
nv n h^ +H 



2c 



E 



exp 



nv n h[ 



d+\a 



■Yi - 1 



and A njX can be rewritten as 



= ^J2 E 



cxp 



ua 



n 



i=l 

i=l ' 4 i 



2 n i / 



cxp 



nv n h^ +H 



(10) 



For proving Lemma 1, we consider two cases: 
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3.1.1 First case: either v n — > oo or led > 1. 



lid 

A Taylor's expansion implies the existence of n between and J+wXi sucn ^ na ^ 

nv n K 



E 



cxp 



ua r . 



Y -1 



nv n hf +lal 



,E(^) + - 



EQ?) + c 



1 / wa r 



6 1™^'°' 



E ( K 3 e<» 



Therefore, 



2 n i 



u) 



with 

i2£J(«) 



6 n ; 



■u 2 a r 



/ (0 M * (*))' [f(x - Kz) - f{x)\ dz. 
i=i ^ yKd 



d+|a| 



K- - 1 



Using (5), one can show that 



ua r 



nvnh'i 



d+\a\ 



< Ci 



l-a(d + 2|a|) 



||^|| c 



where c\ is a positive constant and thus 





it 




l-a(d + 2 


a|) 



and 



2 < exp 2ci 









l-a(d + 2 


a|) 



ll^llc 



\\d^K\\, 



(11) 



(12) 



Noting that E|Yj| 3 < hf \\f\\oo f R d \d^K(z) | 3 dz and using (5) again and (11), there exists a 
positive constant c 2 such that, for n large enough, 



u 3 a 2 



n 3 v n 



n. ^ ' 

c1 |«|||aMic||oo , , 



i=i 



< c 2 



(1 + a(2d + 3|a|)) (1 - a(d + 2|a|)) J w„ 



/ \d^K(z)\ 3 dz (13) 
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which goes to as n — > oo since either v n — > oo or |a| > 1. 

In the same way, in view of (5) and (12), there exists a positive constant c 3 such that, for n 
large enough, 



n 

2a, 



n 1 



E 



1=1 C *:™ 



cxp 



ua 



nvnh'i 



<l+\a\ Yi 



< c 3 



4e ll"l ||a[a 
p l-a(d+2|a|) II" " 



(14) 



which goes to as n — > oo. The combination of (13) and (14) ensures that 

hm sup \RW(u)\=Q. 



Now, since / is continuous, we have lim^oo \ f(x — hiz) — f(x)\ = 0, and thus, by the domi- 
nated convergence theorem, (H4)ii) implies that lim i _ ( . 00 f Rd (d^K(z)) 2 \ f(x — hiz) — f(x)\ dz 



0. Since, in view of (5), the sequence 



u 2 a r , 



1 

X t.d+'. 



d+2\a\ 



is bounded, it follows that 



2n 2 <r~? h" 

i=i n i 

lim \R^\(u)\ = 0. The pointwise convergence (9) then follows. 

In the case / is uniformly continuous, we have lim^oo sup xeK d \f(x — h^z) — f(x)\ = and 
thus, using the same arguments as previously, we obtain lim^^ sup x6R d \Rn}x{u) \ = 0. 
We then deduce that Um^oo sup xe{7 |A„ jX (w) — A x (u)| = which concludes the proof of 
Lemma 1 in this case. 



3.1.2 Second case: \ct\ = and (v n ) = 1. 

Using assumption (H2) and in view of (10), there exists c > such that 

2a r . 



with 



E E 



i=i 

n 



-Y- 



e * 



i=l V 



1 /* d 

a n i=1 JR d 1 J 



R { nl{u) 



2a 
1 



n 

E 



c 2 

1 »,n 



E 



a nh d 1 1 



\K{z) 



Y^hf / le^""~' -l)[f(x-h iZ )-f(x)}dz 

n „ 
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Using the bound (14), we have lim^oo sup x€R d \R ( nl(u)\ =0. 
Since |e* — 1| < |£|e'*', we have 



e" h ? - lj [/(a; - M - /(x) 

/ mz)\\f(x-h t z)-f(x)\ 

n „ 

1=1 "' Rd 

I I n „ 

( e T^II*ll<» + / 1^)1 - M - dz. 

i=i J Rd 



< gl-ad 

71 



In the case / is continuous, we have lim^oo Rn} x {u) = by the dominated convergence 
theorem. 

In the case / is uniformly continuous, set e > and let M > such that 2||/'|| oc , f\\ z \\> M \K( Z )\ dz < 
e/2. We need to prove that for n sufficiently large 

sup / \K(z)\\f(x-hiz)-f(x)\dz < e/2 

x<=R d J \\z\\<M 

which is a straightforward consequence of the uniform continuity of /. It follows from analysis 
considerations that 



limA^H = f(x) I (1 - ad) \ f s- ad ( 

n^oo J Rd [_J 

/(*) 



(f l-ad 



K(z) 



Jq 1 s ad ds Jo 



c^rf^M — 1 — ii 



-uK{z)^j 
K(z)^j dz 



ds 



dz 



ds. 



and thus Lemma 1 is proved. □ 



3.2 Proof of Proposition 2 

To prove Proposition 2, we apply Proposition 1, Lemma 1 and the following result (see 
Puhalskii (1994)). 

Lemma 2 Let (Z n ) be a sequence of real random variables, (u n ) a positive sequence satisfying 
lim^oo u n = +oo, and suppose that there exists some convex non-negative function T defined 
(i.e. finite) on R such that 

\/u e R, lim — logE [exp(uv n Z n )] = F(u). 

If the Legendre transform V ofT is a strictly convex function, then the sequence (Z n ) satisfies 
a LDP of speed (v n ) and good rate function T. 



13 



In our framework, when |a| = and v n = 1, we take Z n = f n (x) — E(/„(x)), v n = Y^=i 
with h n = cn~ a where < a < l/d and T = A x . In this case, the Legendre transform of T = 

is the rate function I x : 1 i— > f(x)(l — ad)I ( j^j(fr^) + iz^) which is strictly convex by 

Proposition 1. Otherwise, we take Z n = v n (d^f n (x) - E (d^f n (x))) , v n = £™ =1 h^^/vl 
and T = A^f; T is then the quadratic rate function J[ a ] >x defined in (2) and thus Proposition 
2 follows. □ 

3.3 Proof of Proposition 3 

In order to prove Proposition 3, we first establish some lemmas. 
Lemma 3 Let : RH > R be the function defined for 5 > as 

\ ii/ii J ( i-«o +ra) # W sl md H=o, 

= < 5(l-a 2 (d+2| Q |) 2 ) 

— — ^r- — otherwise. 

y ii/ik,oo/ B<J [8w^] 

1. sup ugR {tt5 — sup^jy A x (u)} equals gu(o~) and is achieved for u = <p(5) > 0. 

2. sup MgR {— u5 — sup xeU A x (u)} equals gu(—5) and is achieved for u = <f>(—6) < 0. 

Proof of Lemma 3 

We just prove the first part, the proof of the second one being similar. First, let us consider 
the case (v n ) = 1 and |a| = 0. Since e* > l+t (Vt), we have ip(u) > u/(l — ad) and therefore, 

u5- sup A x (u) = u5 - H/l|^oo(l - ad) (ip{u) - U ) 
x&j V Y — ad) 

= 11/11^(1 - ad) [« ( llyn^CX - od) + T^d) " * (0) 

The function w 1— > w5 — sup xe[/ A x (u) has second derivative — 1|/||[/ )00 (1 — ad)ip"(u) < and 
thus it has a unique maximum achieved for 

_ (!l] -lf S 1 

Now, since ip' is increasing and since ip'(0) = 1/(1 — ad), we deduce that Uq > 0. 
In the case lim^oo v n = 00, Lemma 3 is established in the same way by noting that 

u5 - sup A x (u) =u5- sup Af H = u5 - — f ^ ll/lkoo [ [d [a] K(z)] 2 dz. □ 

x&j xeu 2 (1 — a 2 (d + 2\a\) ) 

Lemma 4 

• In the case \a\ = and (v n ) = 1, let (HI) and (H2) hold; 

• In the case either \a\ > 1 and (v n ) = 1, or \a\ > and (i> n ) satisfies (H6), let (HI), 
(H3) and (H4) hold. 



14 



Then for any 5 > 0, 



^ n V L 2H logsupP [v n (dWf n (x) - E (dWf n (x))) > 8} = - 9u (8) 

Urn ^ w ^ +2|Q| log sup P [v n {d [a] f n (x) - E (d™ f n (x))) < -5] = - 9u (-5) 

J™, E n V J d+ 2M lo g^g P ^ \d [a] fn(x) - E (aM /w(a;) ) I > S ] = -~ gu (5). 
Proof of Lemma 4 

n 

Set b n = ^^hf +2 ^/Vn, S n (x) = v n ^n\x), and 8 > 0. In the sequel, A x (u) is denned as in 



i=i 



(9). 

We first note that, for any u > 0, 

P > 5] = P [ e 6 " u5 " (:r) > e Ku& ] 

< e ~ 6 " U<5 ]E |g 6 n u ' S 'n( a; )J 
<- g— 6nw5g6nA n , x (u) 

< g~ fe n («5-Ax («)) g&n (A„,x (it) -Ax («)) 

Therefore, for every w > 0, 

SUpP[S' n (a;) > 5] < e ~ bn ( u5 ~ SUPxeu A ^( u )) e b nsnp xeu \A n , x (u)~A x (u)\ ^ 

xeu 

Similarly, we prove that, for every u < 0, 

SUpP [SV^rr) < -5] < e - bn (- uS ~ sup xeu A *( u )) e bnSup xeu \A n , x (u)-A x (u)\ ^ 

The application of Lemma 3 to (15) and (16) yields 

SUpP [S n (x) > 8] < e ~ bn 9u(5) e l>nSup xeu \A ntX (cf>(5))-A x (4>(8))\ 

xeu 

SUpP [S n (x) < — 8] < e ~ bn 9u(S) e b n sup xeu \A niX (4>(-S))-A x ((p(-S))\ 

and the second part of Lemma 1 provides 

lim sup 1^(0(5)) -A x (0(<J))| = 
limsup|A n)X (0(-(J))-A x (0(-<J))| = 0. 



Consequently, it follows that 

1 



lim sup — log sup P [S n (x) > 8] < —gu{8) 
lim sup -3- log sup P [S n (x) < -5] < -gu{~8) 

n^oo n x£U 
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and thus, setting g v (5) = mm{g u (S), gu(-S)}, 



limsup^-logsupP[|<S' fl (a;)| > 5} < -gu(S). 

n^oo O n x£U 

In order to conclude the proof of Lemma 4, let us note that there exists Xq G U such that 
f( x o) — \\f\\u,oo- The application of Proposition 2 at the point x thus yields 

Um ±\ og F[S n (x ) >5] = - gu {5) 

limi-logP[5 n (x )<-5] = - gu (-8) 

n— >oo Q n 

lim llogP[|5 n (x )| >5] = -M<$). 
The latter relation being due to the straightforward bounds 

max{P [S n (x ) > 5} , P [S n (z ) < -$]} < P [|^(^o)| > 5] 

< 2 max{P [5 n (a;o) > <S] , P [S n (rr ) < -5]}. □ 

Lemma 5 Lei Assumptions (HI), (H3), (H4)i), (H9)i) and (H10) hold and assume that 
either (v n ) = 1 or (H6) holds. 



1. If U is a bounded set, then, for any 5 > 0, we have 

,2 



lim 



logP 



supu n |*W(a;)| >5 



n->oo 1 f l d+2 \ a 

2. If U is an unbounded set, then, for any b > and 5 > 0, 



< -9v($). 



lim sup 



n ,d+2\a\ 



i=l h 



logP 



sup v n \ ^ ] (x) \ > 5 
xeu,\\x\\<w n 



< db-~ gu (S) 



where w n = exp (bJ2"=i hf +2 ^/v^j ■ 



Proof of Lemma 5 



Set p e]0, <5[, let /3 denote the Holder order of d^K, and H^if its corresponding Holder 
norm. Set w n = exp (bY^ =1 h d+2 ^ / 'v£j and 



Rn — 



pn 



2\\dWK\\ H v n Y,; =1 hJ (d+P+lal) 



We begin with the proof of the second part of Lemma 5. There exist N'(n) points of 



Ui^iy^-i ■ ■ ■ ■> Vn'm sucn t na t the ball {x e R d ; ||x|| < u> n } can be covered by the N'(n) balls 



'(«) 
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B ™ = {x G M. d ; \\x — y\ || < and such that N'(n) < 2 ( — ^ ] . Considering only the 
N(n) balls that intersect {x G U; \\x\\ < w n }, we can write 

{xeU; \\x\\ <^}cuS n) ^ (n) . 

For each i e {1,..., N(ri)}, set x\ n ^ G 5^ n U. We then have: 

AT(n) 



< Mn) max P 

l<i<N(n) 



SUP x( . B (n)V n 



> 5 



Now, for any i G {1, . . . , iV(n)} and any x G -Bj- n \ 



Vn \- J_ 

n L^t id+\a\ 
3=1 a j 



d W\ K ( 



x — Xj 



I _ d l°] K ti 



hj 



< v r , 

< V r , 



\x — X 



Mi 



hi 



Hence, we deduce that 



P [™Px 6 ^|N|<«„«n | ^if (x) | > 6} < N(n) ^ P [« B ^L Q] 



> 5-p 



< iV(n) sup P [u n | *W (x) | > 5 - p] . 



Let us at first assume that 



lim sup 



n ,d+2|a| 



log N(n) < db. 



The application of Lemma 4 then yields 



(17) 



lim sup ^ logF [sup xeUtM < Wn v n \^\x)\ > 5] 



< lim sup 



n 

Era , ci+2|a| 
i=l ' 4 i 



log TV (n) -gu(S- p) 



< db- gu(5 - p). 
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Since this inequality holds for any p e]0, S[, Part 2 of Lemma 5 thus follows from the 
continuity of g\j. 

Let us now establish Relation (17). By definition of N(n) and w n , we have logiV(ra) < 
logJV'(n) <rf6Er=i^ +2H /^ + (^+l)log2-rflo gj R n , with 



vt 



n h d+2\a\ 



\ogR n 



v: 



n , d+2\a\ 



logp + logn - log (2\\d [a] K\\ H ) - \ogv n - log ^ h . 



-(d+p+\a\) 



vj'=l 



which goes to zero in view of (H10) and (5). Thus, (17) is proved, and the proof of part 2 of 
Lemma 5 is completed. 

Let us now consider part 1 of Lemma 5. This part is proved by following the same steps 
as for part 2, except that the number N(n) of balls covering U is at most the integer part 
of (A/R n ) d , where A denotes the diameter of U. Relation (17) then becomes 



lim sup 



v; 



En ,d+2\a\ 
i=l n i 



log R n < 



and Lemma 5 is proved. □ 



Lemma 6 Let Assumptions (HI), (H3), (H~4)i) and (Hll)ii) hold. Assume that either 
(y n ) = 1 or (H6), (RIO) and (Hll)i) hold. Moreover assume that d^f is continuous. For 

any b > if we set w n = exp ^&X^=i h d+2 ^ / v £j then, for any p > 0, we have, for n large 
enough, 



V n 1 
sup — > , I I 

xeu,\\ x \\> Wn n h d +W 



E 



d [a] K 



x - Xj 
hi 



< 



Proof of Lemma 6 

We have 

n 1 

n /-^ h d+\a\ 
i=l n i 



d [a] K 



x - Xj 
hi 



Vn 

n 



n „ 

V/ K(z)d [a] f(x-h t z)dz. (18) 
i=i jRd 



Set p > 0. In the case (y n ) = 1, set M such that ||^ [q:] /||oo J\\ z \\ >m \ k ( z )\ dz < p/ 2 5 we have 



n 

i=i 



E 



hi 



< ^ + d [a] f(x) [ \K{z)\dz + -Y] [ \K{z)\\d [a] f{x-h iZ )-d [a] f{x)\dz. 
2 J\\A\< M n i^t A\4<^ 
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Lemma 6 then follows from the fact that d^f fulfills (Hll)ii). As a matter of fact, this 
condition implies that h.mun_ yooxe jjd^f(x) = and that the third term in the right-hand- 
side of the previous inequality goes to as n — > oo (by the dominated convergence). 



Let us now assume that lim^oo v n 



oo; relation (18) can be rewritten as 



v n \ ^ 
n ^ 



i=i h 



d+\a\ 



E 



d [a] K 



x - Xj 



i=i j ii- 
rt 

+-E 



z\\<w n /2 



K(z)d [a] f(x-h lZ )d; 



K(z)d [a] f{x-h iZ )dz 



1 J\\z\\>w„/2 



Set p > 0; on the one hand, we have 
||x|| > w n and \\z\\ < w n /2 



> \\x — hiz\\ > w n (1 — hi/2) 
=>- \\x — hiz\\ > w n /2 for n large enough. 

Set Mf = sup xgRd 1 1 x 1 1 1J d^f(x). Assumption (Hll)ii) implies that, for n sufficiently large, 

n „ n „ 

sup — J2 \K(z)d [a] f(x-h iZ )\dz < sup — J2 \K(z)\Mf\\x-ha 

||x||>u;„ n i=1 J\\z\\<w„/2 \\x\\>w„ n i=1 J\\z\\<w„/2 

< 2^M f [ \K(z)\dz 

< p. 



On the other hand, we note that, in view of assumptions (H10) and (Hll)i), 

n „ 

sup \K(z)d^f(x-h lZ )\dz<2^\\d^fl 

\\x\\>w n n i=1 J\\z\\>w n /2 



| c \K(z)\dz < 



p 



\\z\\>w n /2 * 

(for n large enough). As a matter of fact, we have by assumptions (H6) and (H10), V£ > 



-4 = exp <^ -£logw n 1 



£\ogw n 



0. 



This concludes the proof of Lemma 6. □ 







Since d^K is a bounded function that vanishes at infinity, we have limi^n^oo \^fn (x) 

for every given n > 1. Moreover, since d^K is assumed to be continuous, is continuous, 
and this ensures the existence of a random variable s„ such that 



sup \^ l (x 

xeu 



Lemma 7 Let Assumptions (HI), (H3), (H4)i), (H8)i), (H9)ii) and (H10) hold. Suppose 
either (v n ) = 1 or (H6) and (Ell) hold. For any b > 0, set w n = exp ^X^=i ^h +2 ^ / v nj > 
then, for any 5 > 0, we have 



lim sup 



vi 



n id+2\ 



-logP [||s n || > w n and v n \^ ] (s n )\ > 5] 



< 
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Proof of Lemma 7 

We first note that s n e U and therefore 

||s„|| > w n and v n \¥« ] (s n )\ > 5 



\s n \\ > w n and — 
n 



V" 1 d [a] K ( Sn ~ Xi ) + V 1 d [a] K ( Sn ~ Xi ) 



sjl > w n and — , , , , d^K 
i=i 

sup ^y^_E^-/v 

||x||>u;„,xet/ ' 4 i=1 h i 



hi 
x - Xj 



> 5 



Set p e]0, S[; the application of Lemma 6 ensures that, for n large enough, 
K|| > w n and v n \^ ] (s n )\ > 5 



v 1 
SjJI > w n and — > , i i 

i=l ' t i 



>S-p. 



Set k = sup ^^^^^(a;)! (see Assumption (H9)ii)). We obtain, for n sufficiently large, 



\s n \\ > w n and v n \ w^(s n )\ > 5 



\s n \\ > w n an d 3i e {1, ...,n} such that 



h 



d+\a\ 



d [a] K 



d+\a\ 



hi 



>5-p 



\s n \\ > w n and 3ie {!,..., n} such that K>h] > — ||s n — Xj|| 7 (5 — p) 



|s n || > w n and 3ie{l,...,n} such that | ||s n || — ||Xj|| | < 
|sn|| > w n and Eli e {!,..., n) such that ||Xj|| > ||s n || — 



KV„h 



7— d— \a\ 



5-p 



KV n hi 



-d-\a\ 



5-p 



=>• 3ie{l,...,n} such that > w n (1 - u nji ) with u nyi = w n 1 Vnh, 

Assume for the moment that 



1 7 — d— \c 

1 

I 



lim u n i = 0. 

n^oo 



5-p 



(19) 
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It then follows that 1 — u n ^ > for n sufficiently large; therefore we can deduce that (see 
Assumption (H8)i)): 



P lis 



s n \\>w n and v n \^(s n )\>5] < ^ P [\\X t f > w$ (1 - u n ,f 

i=l 

n 

< E{\\Xi\\*)v>?(i -un,r* 

i=l 

< nE (||Xi|| ? ) max (l-u n ^ 

v ' l<i<n 

Consequently, 

^logP [||s n || > w n and v n \^ ] (s n )\ > 5] 



En id+2\a\ 
i=l n i 



< 



v; 



n ,d+2\a\ 



En 7 ■ 
i=l k 



\r^n ,d+2\a\ 

logn + logE (WX^) - fc^L_* Clog max (1 - u n . 



Ki<n 



and, thanks to assumption (H10), it follows that 

limsup l gP[K|| >w n and v n \^\s n )\>5] < 

Let us now prove relation (19). We expand 



u nA = exp -b 



En i a 
i=i h i 



d+2\a\ 



1 vllogv n ~f-d-\a\ vllog(hi 



n 7 d+2\a\ 



K \ T 



S-p 



and assumptions (H6) and (H10) ensure that lim^oo u n>i = and thus Lemma 7 is proved. □ 



Proof of Proposition 3 

Let us at first note that the lower bound 



lim inf n 

n-»oo h d+2 \ a \ 



En 7 a 
i=i K 



logP 



supw n W^{x)\ > 5 

xeu 



> -~9u{8) 



(20) 



follows from the application of Proposition 2 at a point x E U such that f(xo) = \\j \\u,oo- 
In the case U is bounded, Proposition 3 is thus a straightforward consequence of (20) and 
of the first part of Lemma 5. Let us now consider the case U is unbounded. 
Set 5 > and, for any b > set w n = exp (bY^ = \ h^ +2 ^ /v£j. Since, by definition of s n , 



supv n \^(x)\ >5 



< P 



sup v n \^(x)\>5 



x£U,\\x\\<w n 



+ P[W >w n and v n \vW(s n )\ >5] 
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it follows from Lemmas 5 and 7 that 

,2 



lim sup 



vz 



n id+2\a\ 



logP 



supv n \^\x) \ > 5 

x&J 



< max{— b£; db — gu($)} 



and consequently 



lim sup 



vz 



En 7 d+2|« 
i=l n i 



logP 



supu n K Q ](a;)| > 5 

x&J 



< inf max{— &£; db — <7[/(5)}. 



fe>0 



Since the infimum in the right-hand-side of the previous bound is achieved for b = gu{8) / (£ + d) 
and equals —£gu(d)/ (£ + we obtain the upper bound 



lim sup 



logP 



En ,d+2\a 
i=l n i 

which concludes the proof of Proposition 3. □ 



supu n |*W(a;)| >5 

x&J 



< 



z + d 



~gu(S) 



3.4 Proof of Proposition 4 

Let us set g = d^f, D^g (j £ {1, . . . , q}) the j-th differential of g, y — (y±, . . . , y^) E 
and yd) = (y, . . . , y) e (IR d ) J . With these notations, 

Dig{x){y^) = £ ^\ >a » yr . . . iff 



aiH h«d=j 



,. dyT ...dy d 



By successive integrations by parts (and using the fact that the partial derivatives of K 
vanish at infinity, see Assumptions (H4)i)), we have 



E 



d^f n {x) 



d+\a\- 



lA l ^ 
— / — t-^tE 
n , h 

i=l ,h i 



1 n 1 

n ^ h d +\ a \ 
i=i 

lA i_ 



d [a] K 



X - X; 



hi 
hi 



f(y)dy 



= -^2 K(y)g(x - Ky)dy. 

Hence, using assumption (H7)i) and the fact that d^f is g-times differentiable, it comes 

E[d^f n (x)] -d^f(x) 
1 n f 

= ~ Y] / K(y) [g(x - tuy) - g(x)] dy 

n i=1 JR d 



n f—' .La 



i=i 



y(x - ^y) - y(x) - ^-^h\D j g(x)(y 



0> 



dy. (21) 
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Let us set 

Ui(x) = [ K(y) 

JR d 



g(x - My) - g(x) - Z%1 g{x){y^) 



hi 



dy and 



Uoo(x) 



D«g(x)(y^)K(y)dy 



We clearly have 

lim Ui(x) = Uooi^x) 

i— +00 

and therefore, Ve > 0, 3i e K such that Vi > i , \Ui(x) — Uoo(x)\ < e. 
• K Si = 00 i then 

n [E(^/ n (x))-9[ Q ]/(x)] -CU*) 
- C/oo(a?) 



Er=i ^ 



< 



ESri 1 ^ I W - + EL„ ^ I W - M*) 



EILA 



< 2e. 



If Ei < 00 ' we can WI "ite 



n 



Er=i k 



[E (9 [a] /„(a;)) - cH/(:r)] 



Er=i ^ 



(22) 



In view of (22), for x fixed and for all i G N, the sequence (£/j(x))i is bounded and thus Part 
1 of Proposition 4 is completed. Let us now prove Part 2. 

Since the bracketed term in (21) is bounded by sup xeR d ||Z) 9 p(a;)|| = M q (see Assumption 
(H7)iii)), Part 2 follows. □ 



3.5 Proof of Proposition 1 



• Since |e* — 1| < \t\ e'*' Vt G M, and thanks to the boundedness and integrability of K, 
we have 



/ 



-ad 



[0,1] xR d 



dsdz < 



\U\ H 

g 1 — ad 



1 — ad 



I s~ ad \K(z)\ dsdz < 00 

J[0,l]xR d 



which ensures the existence of ijj. It is straightforward to check that ip is twice differ- 
entiable, with 



i)\u) = [ K(z)e sad ^ K(z) dsdz, 

1 — ad i[ 0i i]xR d 



1 



(1 - ad) J[q,i]xi 
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s ad {K{z)fe sad T^ K ^dsdz. 



Since ip"(u) > Vm G R, ^' is increasing on R, and tp is strictly convex on R. It follows 
that its Cramer transform J is a good rate function on R (see Dembo and Zeitouni 
(1998)) and (i) of Proposition 1 is proved. 

• Let us now assume that A(<S'_) = 0. We then have 

lim ip'(u) = and lim ip'(u) = +oo, 

so that the range of ip' is ]0, +oo[. Moreover lim u _ ) ._ 00 ip(u) = —X(S + )/(l — ad) (which 
can be — oo). This implies in particular that 1(0) = X(S + )/(l — ad). Now, when 
t < 0, liniu^-oo (ut — ip(u)) = +oo, and I(t) = +oo. Since ip' is increasing with range 
]0, +oo[, when t > 0, sup u (wt — ip(u)) is reached for uo(t) such that ip'(uo(t)) = t, i.e. 
for uo(t) = (■0') _1 (t); this prove (3). (Note that, since tp"(t) > 0, the function 1 1— > u (t) 
is differentiate on ]0, +oo[). Now, differentiating (3), we have 

I'(t) = uo(t)+tu' (t)-^(u (t))u' (t) 

= W)-\t) + K(t) - tu' (t) 

Since (ip')~ l is an increasing function on ]0, +oo[, it follows that / is strictly convex on 
]0,+oo[ (and differentiable). Thus (ii) is proved. 
Now, since X(S-) = 0, ^'(0) = 1/(1 - ad) > 0; we have 

/'(*)= (^')~ 1 (t)=0 iP'(0)=t t = 1/(1 -ad). 

Then I' (1/(1 — ad)) = 0, and I (1/(1 — ad)) = is the unique global minimum of / 
on ]0, +oo[. This proves (iv) when \(SJ) = 0. 

• Assume that \(S ) > 0. In this case, ip' can be rewritten as 

4>'(u) = — *— - [ K(z)e sad ^ K ^dsdz 
1 - ad J[o,i]x(R d ns+) 

+—*— [ K(z)e sad T^ K ^dsdz 
1 - ad J[o,i] x (R d nS-) 

and we have 

lim ip'(u) = — oo and lim ip'(u) = +oo 

so that the range of ip' is R in this case. The proof of (Hi) and the case X(S-) > 
of (iv) follows the same lines as previously, except that, in the present case, (ip')" 1 is 
defined on R, and not only on ]0, +oo[. □ 

References 

[1] Ahmad, LA. and Lin, P. (1976), Nonparametric sequential estimation of a multiple 
regression function. Bull. Math. Satist. 17, pp. 63-75. 



24 



[2] Carroll, R.J. (1976), On sequential density estimation. Z. Wahrscheinlichkeitstheorie 
und Verw. Gebiete 36, pp. 137-151. 

[3] Davies, H.I. (1973), Strong consistency of a sequential estimator of a probability density 
function. Bull. Math. Statist. 15, pp. 49-54. 

[4] Dembo, A. and Zeitouni, O. (1998), Large deviations techniques and applications. 
Springer, Applications of mathematics, New- York. 

[5] Devroye, L. (1979), On the pointwise and integral convergence of recursive kernel esti- 
mates of probability densities. Utilitas Math. 15, pp. 113-128. 

[6] Feller (1970), An introduction to probability theory and its applications. Second edition 
Volume II, Wiley. 

[7] Gao, F. (2003), Moderate deviations and large deviations for kernel density estimators. 
Journal of Theoritical Probab. 16, pp. 401-418. 

[8] Louani, D. (1998), Large deviations limit theorems for the kernel density estimator. 
Scand. J. Statist. 25, pp. 243-253. 

[9] Menon, V.V., Prasad, B. and Singh, R.S. (1984), Non-parametric recursive estimates 
of a probability density function and its derivatives. Journal of Statistical Planning and 
inference 9, pp. 73-82. 

[10] Mokkadem, A., Pelletier, M. and Worms, J. (2005), Large and moderate deviations 
principles for kernel estimation of a multivariate density and its partial derivatives, to 
appear in Austral. J. Satist. 

[11] Parzen, E. (1962), On estimation of a probability density function and mode. Ann. 
Math. Statist. 33, pp. 1065-1076. 

[12] Puhalskii, A. A. (1994), The method of stochastic exponentials for large deviations. 
Stoch. Proc. and their Appl. 54, pp. 45-70. 

[13] Rosenblatt, M. (1956), Remarks on some nonparametric estimates of density function. 
Ann. Math. Statist. 27, pp. 832-837. 

[14] Roussas, G.G. (1992), Exact rates of almost sure convergence of a recursive kernel 
estimate of a probability density function: Application to regression and hazard rate 
estimation. J. Nonparametric Statist. 1, pp. 171-195. 

[15] Wegman, E.J. (1972), Nonparametric probability density estimation. A comparison of 
density estimation methods. J. Statist. Comput. Simulation 1, pp. 225-245. 

[16] Wegman, E.J. and Davies, H.I. (1979), Remarks on some recursive estimators of a 
probability density. Annals of Satistics 7, pp. 316-327. 

[17] Wertz, W. (1985), Sequential and recursive estimators of the probability density. Statis- 
tics 16, pp. 277-295. 



25 



[18] Wolverton, C.T. and Wagner T.J. (1969), Asymptotically optimal discriminant func- 
tions for pattern classification. IEEE Trans. Inform. Theory 15, pp. 258-265. 

[19] Worms, J. (2001), Moderate and large deviations of some dependent variables, Part II: 
some kernel estimators, Math. Methods Statist. 10, pp. 161-193. 

[20] Yamato, H. (1971), Sequential estimation of a continuous probability density function 
and mode. Bull. Math. Statist. 14, pp. 1-12. 



26 



